{
    "metadata": {
        "kernelspec": {
            "name": "python3",
            "display_name": "Python 3"
        },
        "language_info": {
            "name": "python",
            "version": "3.6.6",
            "mimetype": "text/x-python",
            "codemirror_mode": {
                "name": "ipython",
                "version": 3
            },
            "pygments_lexer": "ipython3",
            "nbconvert_exporter": "python",
            "file_extension": ".py"
        }
    },
    "nbformat_minor": 2,
    "nbformat": 4,
    "cells": [
        {
            "cell_type": "markdown",
            "source": [
                "![Microsoft](https://raw.githubusercontent.com/microsoft/azuredatastudio/master/src/sql/media/microsoft-small-logo.png)\n",
                " \n",
                "## Deploy SQL Server 2019 Big Data Cluster on an existing cluster deployed using kubeadm\n",
                " \n",
                "This notebook walks through the process of deploying a <a href=\"https://docs.microsoft.com/sql/big-data-cluster/big-data-cluster-overview?view=sqlallproducts-allversions\">SQL Server 2019 Big Data Cluster</a> on an existing kubeadm cluster.\n",
                " \n",
                "* Follow the instructions in the **Prerequisites** cell to install the tools if not already installed.\n",
                "* The **Required information** will check and prompt you for password if it is not set in the environment variable. The password can be used to access the cluster controller, SQL Server, and Knox.\n",
                "\n",
                "<span style=\"color:red\"><font size=\"3\">Please press the \"Run Cells\" button to run the notebook</font></span>"
            ],
            "metadata": {
                "azdata_cell_guid": "23954d96-3932-4a8e-ab73-da605f99b1a4"
            }
        },
        {
            "cell_type": "markdown",
            "source": [
                "### **Prerequisites** \n",
                "Ensure the following tools are installed and added to PATH before proceeding.\n",
                " \n",
                "|Tools|Description|Installation|\n",
                "|---|---|---|\n",
                "|kubectl | Command-line tool for monitoring the underlying Kuberentes cluster | [Installation](https://kubernetes.io/docs/tasks/tools/install-kubectl/#install-kubectl-binary-using-native-package-management) |\n",
                "|azdata | Command-line tool for installing and managing a Big Data Cluster |[Installation](https://docs.microsoft.com/en-us/sql/big-data-cluster/deploy-install-azdata?view=sqlallproducts-allversions) |"
            ],
            "metadata": {
                "azdata_cell_guid": "1d7f4c6a-0cb8-4ecc-81c8-544712253a3f"
            }
        },
        {
            "cell_type": "markdown",
            "source": [
                "### **Setup**"
            ],
            "metadata": {
                "azdata_cell_guid": "a31f9894-903f-4e19-a5a8-6fd888ff013b"
            }
        },
        {
            "cell_type": "code",
            "source": [
                "import pandas,sys,os,json,html,getpass,time\n",
                "pandas_version = pandas.__version__.split('.')\n",
                "pandas_major = int(pandas_version[0])\n",
                "pandas_minor = int(pandas_version[1])\n",
                "pandas_patch = int(pandas_version[2])\n",
                "if not (pandas_major > 0 or (pandas_major == 0 and pandas_minor > 24) or (pandas_major == 0 and pandas_minor == 24 and pandas_patch >= 2)):\n",
                "    sys.exit('Please upgrade the Notebook dependency before you can proceed, you can do it by running the \"Reinstall Notebook dependencies\" command in command palette (View menu -> Command Palette…).')\n",
                "def run_command(command):\n",
                "    print(\"Executing: \" + command)\n",
                "    !{command}\n",
                "    if _exit_code != 0:\n",
                "        sys.exit(f'Command execution failed with exit code: {str(_exit_code)}.\\n\\t{command}\\n')\n",
                "    print(f'Successfully executed: {command}')"
            ],
            "metadata": {
                "azdata_cell_guid": "26fa8bc4-4b8e-4c31-ae11-50484821cea8",
                "tags": [
					"hide_input"
				]
            },
            "outputs": [],
            "execution_count": 1
        },
        {
            "cell_type": "markdown",
            "source": [
                "### **Set variables**\n",
                "Generated by Azure Data Studio using the values collected in the Deploy Big Data Cluster wizard"
            ],
            "metadata": {
                "azdata_cell_guid": "e70640d0-6059-4cab-939e-e985a978c0da"
            }
        },
        {
            "cell_type": "markdown",
            "source": [
                "### **Check dependencies**"
            ],
            "metadata": {
                "azdata_cell_guid": "869d0397-a280-4dc4-be76-d652189b5131"
            }
        },
        {
            "cell_type": "code",
            "source": [
                "run_command('kubectl version --client=true')\n",
                "run_command('azdata --version')"
            ],
            "metadata": {
                "azdata_cell_guid": "c38afb67-1132-495e-9af1-35bf067acbeb",
                "tags": [
                    "hide_input"
                ]
            },
            "outputs": [],
            "execution_count": 0
        },
        {
            "cell_type": "markdown",
            "source": [
                "### **Required information**"
            ],
            "metadata": {
                "azdata_cell_guid": "7b383b0d-5687-45b3-a16f-ba3b170c796e"
            }
        },
        {
            "cell_type": "code",
            "source": [
                "invoked_by_wizard = \"AZDATA_NB_VAR_BDC_ADMIN_PASSWORD\" in os.environ\n",
                "if invoked_by_wizard:\n",
                "    mssql_password = os.environ[\"AZDATA_NB_VAR_BDC_ADMIN_PASSWORD\"]\n",
                "    if mssql_auth_mode == \"ad\":\n",
                "        mssql_domain_service_account_password = os.environ[\"AZDATA_NB_VAR_BDC_AD_DOMAIN_SVC_PASSWORD\"]\n",
                "else:\n",
                "    mssql_password = getpass.getpass(prompt = 'SQL Server 2019 Big Data Cluster controller password')\n",
                "    if mssql_password == \"\":\n",
                "        sys.exit(f'Password is required.')\n",
                "    confirm_password = getpass.getpass(prompt = 'Confirm password')\n",
                "    if mssql_password != confirm_password:\n",
                "        sys.exit(f'Passwords do not match.')\n",
                "    if mssql_auth_mode == \"ad\":\n",
                "        mssql_domain_service_account_password = getpass.getpass(prompt = 'Domain service account password')\n",
                "        if mssql_domain_service_account_password == \"\":\n",
                "            sys.exit(f'Domain service account password is required.')\n",
                "print('You can also use the controller password to access Knox and SQL Server.')"
            ],
            "metadata": {
                "azdata_cell_guid": "b5970f2b-cf13-41af-b0a2-5133d840325e",
                "tags": [
                    "hide_input"
                ]
            },
            "outputs": [],
            "execution_count": 3
        },
        {
            "cell_type": "markdown",
            "source": [
                "### **Set and show current context**"
            ],
            "metadata": {
                "azdata_cell_guid": "6456bd0c-5b64-4d76-be59-e3a5b32697f5"
            }
        },
        {
            "cell_type": "code",
            "source": [
                "run_command(f'kubectl config use-context {mssql_cluster_context}')\n",
                "run_command('kubectl config current-context')"
            ],
            "metadata": {
                "azdata_cell_guid": "a38f8b3a-f93a-484c-b9e2-4eba3ed99cc2"
            },
            "outputs": [],
            "execution_count": 0
        },
        {
            "cell_type": "markdown",
            "source": [
                "### **Create deployment configuration files**"
            ],
            "metadata": {
                "azdata_cell_guid": "6d78da36-6af5-4309-baad-bc81bb2cdb7f"
            }
        },
        {
            "cell_type": "code",
            "source": [
                "mssql_target_profile = 'ads-bdc-custom-profile'\n",
                "if not os.path.exists(mssql_target_profile):\n",
                "    os.mkdir(mssql_target_profile)\n",
                "bdcJsonObj = json.loads(bdc_json)\n",
                "controlJsonObj = json.loads(control_json)\n",
                "bdcJsonFile = open(f'{mssql_target_profile}/bdc.json', 'w')\n",
                "bdcJsonFile.write(json.dumps(bdcJsonObj, indent = 4))\n",
                "bdcJsonFile.close()\n",
                "controlJsonFile = open(f'{mssql_target_profile}/control.json', 'w')\n",
                "controlJsonFile.write(json.dumps(controlJsonObj, indent = 4))\n",
                "controlJsonFile.close()\n",
                "print(f'Created deployment configuration folder: {mssql_target_profile}')"
            ],
            "metadata": {
                "azdata_cell_guid": "3110ab23-ecfc-4e36-a1c5-28536b7edebf",
                "tags": [
                    "hide_input"
                ]
            },
            "outputs": [],
            "execution_count": 6
        },
        {
            "cell_type": "markdown",
            "source": [
                "### **Create SQL Server 2019 Big Data Cluster**"
            ],
            "metadata": {
                "azdata_cell_guid": "7d56d262-8cd5-49e4-b745-332c6e7a3cb2"
            }
        },
        {
            "cell_type": "code",
            "source": [
                "print (f'Creating SQL Server 2019 Big Data Cluster: {mssql_cluster_name} using configuration {mssql_target_profile}')\n",
                "os.environ[\"ACCEPT_EULA\"] = 'yes'\n",
                "os.environ[\"AZDATA_USERNAME\"] = mssql_username\n",
                "os.environ[\"AZDATA_PASSWORD\"] = mssql_password\n",
                "if mssql_auth_mode == \"ad\":\n",
                "    os.environ[\"DOMAIN_SERVICE_ACCOUNT_USERNAME\"] = mssql_domain_service_account_username\n",
                "    os.environ[\"DOMAIN_SERVICE_ACCOUNT_PASSWORD\"] = mssql_domain_service_account_password\n",
                "if os.name == 'nt':\n",
                "    print(f'If you don\\'t see output produced by azdata, you can run the following command in a terminal window to check the deployment status:\\n\\t{os.environ[\"AZDATA_NB_VAR_KUBECTL\"]} get pods -n {mssql_cluster_name} ')\n",
                "run_command(f'azdata bdc create -c {mssql_target_profile}')"
            ],
            "metadata": {
                "azdata_cell_guid": "0a743e88-e7d0-4b41-b8a3-e43985d15f2b",
                "tags": [
                    "hide_input"
                ]
            },
            "outputs": [],
            "execution_count": 7
        },
        {
            "cell_type": "markdown",
            "source": [
                "### **Login to SQL Server 2019 Big Data Cluster**"
            ],
            "metadata": {
                "azdata_cell_guid": "7929fd90-324d-482a-a101-ae29cb183691"
            }
        },
        {
            "cell_type": "code",
            "source": [
                "run_command(f'azdata login -n {mssql_cluster_name}')"
            ],
            "metadata": {
                "azdata_cell_guid": "3a49909b-e09e-4e62-a825-c39de2cffc94",
                "tags": [
                    "hide_input"
                ]
            },
            "outputs": [],
            "execution_count": 8
        },
        {
            "cell_type": "markdown",
            "source": [
                "### **Show SQL Server 2019 Big Data Cluster endpoints**"
            ],
            "metadata": {
                "azdata_cell_guid": "038e801a-a393-4f8d-8e2d-97bc3b740b0c"
            }
        },
        {
            "cell_type": "code",
            "source": [
                "from IPython.display import *\n",
                "pandas.set_option('display.max_colwidth', -1)\n",
                "cmd = f'azdata bdc endpoint list'\n",
                "cmdOutput = !{cmd}\n",
                "endpoints = json.loads(''.join(cmdOutput))\n",
                "endpointsDataFrame = pandas.DataFrame(endpoints)\n",
                "endpointsDataFrame.columns = [' '.join(word[0].upper() + word[1:] for word in columnName.split()) for columnName in endpoints[0].keys()]\n",
                "display(HTML(endpointsDataFrame.to_html(index=False, render_links=True)))"
            ],
            "metadata": {
                "azdata_cell_guid": "2a8c8d5d-862c-4672-9309-38aa03afc4e6",
                "tags": [
                    "hide_input"
                ]
            },
            "outputs": [],
            "execution_count": 9
        },
        {
            "cell_type": "markdown",
            "source": [
                "### **Connect to SQL Server Master instance in Azure Data Studio**\n",
                "Click the link below to connect to the SQL Server Master instance of the SQL Server 2019 Big Data Cluster."
            ],
            "metadata": {
                "azdata_cell_guid": "0bd809fa-8225-4954-a50c-da57ea167896"
            }
        },
        {
            "cell_type": "code",
            "source": [
                "sqlEndpoints = [x for x in endpoints if x['name'] == 'sql-server-master']\n",
                "if sqlEndpoints and len(sqlEndpoints) == 1:\n",
                "    connectionParameter = '{\"serverName\":\"' + sqlEndpoints[0]['endpoint'] + '\",\"providerName\":\"MSSQL\",\"authenticationType\":\"SqlLogin\",\"userName\":' + json.dumps(mssql_username) + ',\"password\":' + json.dumps(mssql_password) + '}'\n",
                "    display(HTML('<br/><a href=\"command:azdata.connect?' + html.escape(connectionParameter)+'\"><font size=\"3\">Click here to connect to SQL Server Master instance</font></a><br/>'))\n",
                "else:\n",
                "    sys.exit('Could not find the SQL Server Master instance endpoint.')"
            ],
            "metadata": {
                "azdata_cell_guid": "d591785d-71aa-4c5d-9cbb-a7da79bca503",
                "tags": [
                    "hide_input"
                ]
            },
            "outputs": [],
            "execution_count": 10
        }
    ]
}
